Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add example-client docker image #2705

Merged
merged 5 commits into from
Oct 21, 2024

Conversation

vankichi
Copy link
Contributor

@vankichi vankichi commented Oct 16, 2024

Description

SSIA

Related Issue

Versions

  • Vald Version: v1.7.13
  • Go Version: v1.23.2
  • Rust Version: v1.81.0
  • Docker Version: v27.3.1
  • Kubernetes Version: v1.31.1
  • Helm Version: v3.16.2
  • NGT Version: v2.2.4
  • Faiss Version: v1.9.0

Checklist

Special notes for your reviewer

Summary by CodeRabbit

  • New Features

    • Introduced a new GitHub Actions workflow for automating Docker image builds for the example client.
    • Added support for building and packaging an example client target.
    • New Docker image build targets for the example client have been implemented.
  • Improvements

    • Enhanced the Makefile with new variables and functions to streamline the build process for the example client.
    • Updated distance calculation method in configuration for improved feature vector processing.

These changes improve the efficiency and automation of the build and deployment processes for the example client application.

Signed-off-by: vankichi <kyukawa315@gmail.com>
Copy link
Contributor

coderabbitai bot commented Oct 16, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

This pull request introduces several changes aimed at enhancing the automation and build processes for a Docker image related to an example client. A new GitHub Actions workflow file is added to automate Docker image builds triggered by specific events. Modifications are made to various Makefiles to support new targets and functions for building the example client and its Docker image. Additionally, a Dockerfile is provided to define the multi-stage build process for the client application.

Changes

File Change Summary
.github/workflows/dockers-example-client-image.yaml New workflow added to automate Docker image builds for the example client, triggered by specific events.
Makefile New variable EXAMPLE_CLIENT_IMAGE added; AGENT_NGT_IMAGE moved to a new position.
Makefile.d/build.mk New build rule for example/client/client target added; updated dependencies for binary targets.
Makefile.d/docker.mk New phony targets docker/name/example-client and docker/build/example-client added.
Makefile.d/functions.mk New function go-example-build added to facilitate building Go projects in a specified directory.
dockers/example/client/Dockerfile Dockerfile defined for multi-stage build of the client application, specifying build arguments and dependencies.
example/helm/values.yaml Updated distance_type parameter in agent.ngt section from cos to l2.

Possibly related PRs

Suggested labels

priority/low, size/M, actions/e2e-deploy

Suggested reviewers

  • kpango
  • hlts2
  • kmrmt

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between b8beaa7 and a160aa0.

📒 Files selected for processing (1)
  • Makefile.d/functions.mk (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • Makefile.d/functions.mk

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@vankichi vankichi requested review from kmrmt and kpango October 16, 2024 07:42
@vdaas-ci
Copy link
Collaborator

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

kpango
kpango previously approved these changes Oct 16, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (4)
.github/workflows/dockers-example-client-image.yaml (2)

21-70: Comprehensive triggers, but consider optimizing path filters.

The workflow triggers are well-defined, covering various scenarios including pushes to important branches and tags, as well as pull requests that modify relevant files. The use of pull_request_target is appropriate for handling contributions from forks securely.

However, the path filters for pull requests and pull_request_target events are quite extensive. While this ensures that the workflow runs when necessary, it might lead to frequent triggering of the workflow, potentially increasing CI load and slowing down development.

Consider reviewing the path filters to ensure they only include files that directly affect the example-client Docker image. For instance, some internal paths like internal/** might not be necessary unless they contain code specifically used by the example client.


71-76: LGTM: Job definition is concise and utilizes reusable workflow.

The job definition effectively uses a reusable workflow, which is a good practice for maintaining consistency across similar jobs and reducing duplication. The use of a parameter for the target allows for flexibility, and inheriting secrets ensures secure access to necessary credentials.

Consider adding a notification step or a separate job that runs if the build fails. This could help alert the team quickly if there are issues with the Docker image build process. For example:

jobs:
  build:
    uses: ./.github/workflows/_docker-image.yaml
    with:
      target: example-client
    secrets: inherit

  notify-on-failure:
    needs: build
    if: failure()
    runs-on: ubuntu-latest
    steps:
      - name: Send notification
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ job.status }}
          text: 'Docker image build for example-client failed'
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

This example uses Slack for notifications, but you can adapt it to your preferred notification method.

Makefile.d/docker.mk (1)

388-399: LGTM: Correctly implemented example-client Docker targets.

The new targets docker/name/example-client and docker/build/example-client are well-implemented and consistent with other targets in the Makefile. They correctly use the appropriate variables and include necessary build arguments.

For consistency with other targets, consider adding a comment above the docker/build/example-client target to describe its purpose, similar to other build targets in the file.

You could add a comment like this above the docker/build/example-client target:

 .PHONY: docker/build/example-client
+## build example-client image
 docker/build/example-client:
Makefile.d/functions.mk (1)

94-131: LGTM with suggestions for improvement

The go-example-build function is well-structured and serves its purpose of building Go projects in a specified directory. However, there are a few areas where it could be improved:

  1. Error handling: Consider adding error checking after the go build command to ensure the build was successful before attempting to print the version.

  2. Main file location: The function assumes main.go is in the root of the specified directory. Consider making this more flexible by searching for the main file or allowing it to be specified as a parameter.

  3. Version printing: The -version flag might not be available for all binaries. Consider making this step optional or handling potential errors.

Here's a suggested improvement for error handling and flexible main file location:

 define go-example-build
 	echo $(GO_SOURCES_INTERNAL)
 	echo $(PBGOS)
 	echo $(shell find $(ROOTDIR)/$1 -type f -name '*.go' -not -name '*_test.go' -not -name 'doc.go')
 	cd $(ROOTDIR)/$1 && \
+	MAIN_FILE=$$(find . -name 'main.go' | head -n 1) && \
+	if [ -z "$$MAIN_FILE" ]; then \
+		echo "Error: main.go not found in $(ROOTDIR)/$1" >&2; \
+		exit 1; \
+	fi && \
 	CFLAGS="$(CFLAGS)" \
 	CXXFLAGS="$(CXXFLAGS)" \
 	CGO_ENABLED=$(CGO_ENABLED) \
 	CGO_CXXFLAGS="$3" \
 	CGO_FFLAGS="$3" \
 	CGO_LDFLAGS="$3" \
 	GO111MODULE=on \
 	GOARCH=$(GOARCH) \
 	GOOS=$(GOOS) \
 	GOPRIVATE=$(GOPRIVATE) \
 	GO_VERSION=$(GO_VERSION) \
 	go build \
 		--ldflags "-w $2 \
 		-extldflags '$3' \
 		-X '$(GOPKG)/internal/info.AlgorithmInfo=$5' \
 		-X '$(GOPKG)/internal/info.BuildCPUInfoFlags=$(CPU_INFO_FLAGS)' \
 		-X '$(GOPKG)/internal/info.BuildTime=$(DATETIME)' \
 		-X '$(GOPKG)/internal/info.CGOEnabled=$(if $(filter 1,$(strip $(CGO_ENABLED))),true,false)' \
 		-X '$(GOPKG)/internal/info.GitCommit=$(GIT_COMMIT)' \
 		-X '$(GOPKG)/internal/info.GoArch=$(GOARCH)' \
 		-X '$(GOPKG)/internal/info.GoOS=$(GOOS)' \
 		-X '$(GOPKG)/internal/info.GoVersion=$(GO_VERSION)' \
 		-X '$(GOPKG)/internal/info.Version=$(VERSION)' \
 		-buildid=" \
 		-modcacherw \
 		-mod=readonly \
 		-a \
 		-tags "osusergo netgo static_build$4" \
 		-trimpath \
 		-o $(ROOTDIR)/$6 \
-		main.go
-	$6 -version
+		$$MAIN_FILE && \
+	if [ $$? -eq 0 ]; then \
+		$(ROOTDIR)/$6 -version || true; \
+	else \
+		echo "Error: Build failed" >&2; \
+		exit 1; \
+	fi
 endef

These changes will make the function more robust and flexible.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 0a8b332 and e89e1ae.

⛔ Files ignored due to path filters (1)
  • hack/docker/gen/main.go is excluded by !**/gen/**
📒 Files selected for processing (6)
  • .github/workflows/dockers-example-client-image.yaml (1 hunks)
  • Makefile (2 hunks)
  • Makefile.d/build.mk (4 hunks)
  • Makefile.d/docker.mk (3 hunks)
  • Makefile.d/functions.mk (1 hunks)
  • dockers/example/client/Dockerfile (1 hunks)
🧰 Additional context used
🔇 Additional comments (11)
.github/workflows/dockers-example-client-image.yaml (1)

1-20: LGTM: File header and metadata are well-structured.

The copyright notice, edit warning, and workflow name are all correctly formatted and provide clear information about the file's purpose and origin.

Makefile.d/build.mk (5)

34-34: LGTM: Example client added to binary build target

The addition of example/client/client to the binary/build target is appropriate and consistent with the existing structure. This ensures that the example client is built along with other binaries in the project.


127-127: LGTM: Example client artifact added to zip target

The addition of artifacts/vald-example-client-$(GOOS)-$(GOARCH).zip to the binary/build/zip target is appropriate and consistent with the existing structure. This ensures that the example client is included in the zip artifacts along with other binaries in the project.


201-204: LGTM: New rule for example client zip artifact

The new rule for creating the vald-example-client-$(GOOS)-$(GOARCH).zip artifact is well-structured and consistent with other artifact creation rules in the Makefile. It correctly ensures the target directory exists and uses the appropriate zip command to create the artifact.


Line range hint 1-204: Summary: Example client successfully integrated into build process

The changes to Makefile.d/build.mk successfully integrate the example client into the build and packaging process. The additions are consistent with the existing structure and follow established patterns. Key points:

  1. The example client is now part of the main binary build target.
  2. A new build rule for the example client is added, using CGO and appropriate flags.
  3. The example client is included in the zip artifacts target.
  4. A new rule for creating the example client zip artifact is added.

These changes appear to be well-implemented and maintain the consistency of the Makefile. The only outstanding question is about the go-example-build function, which may require further clarification.


107-109: LGTM: New build rule for example client

The new build rule for example/client/client is well-structured and consistent with other CGO-enabled builds in the Makefile. It correctly sets CGO_ENABLED = 1 and uses appropriate linkage and flags.

Could you please clarify the following:

  1. What is the go-example-build function, and how does it differ from the standard go-build function?
  2. Is the dependency on HDF5 necessary for the example client?

To verify the go-example-build function, please run:

✅ Verification successful

Clarifications on go-example-build Function and HDF5 Dependency

The go-example-build function is defined in Makefile.d/functions.mk. It serves as a custom wrapper around the standard go build command, incorporating additional configurations to handle CGO settings and specific linker flags required for building the example client.

Regarding the HDF5 dependency, the inclusion of HDF5_LDFLAGS and HDF5_VERSION in the build command indicates that the example client depends on the HDF5 library. This dependency is necessary for the client's functionality related to HDF5 operations.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for the definition of go-example-build function
rg --type make "go-example-build\s*=\s*" -A 10

Length of output: 48


Script:

#!/bin/bash
# Search for all occurrences of go-example-build in makefiles
rg --type make "go-example-build" -A 10

Length of output: 1199

Makefile.d/docker.mk (3)

32-32: LGTM: Correctly integrated example-client into the build process.

The addition of docker/build/example-client to the docker/build target ensures that the example-client Docker image is built along with other images. This change is consistent with the existing structure and enhances the build system.


60-60: LGTM: Correctly integrated example-client into parallel build process.

The addition of docker/build/example-client to the xpanes command ensures that the example-client Docker image is built in parallel with other images. This change is consistent with the existing structure and improves build efficiency.


Line range hint 1-399: Overall: Well-implemented changes for example-client Docker image.

The changes to integrate the example-client Docker image into the build process are well-implemented and consistent with the existing structure of the Makefile. All necessary targets have been added correctly, and the changes enhance the build system by including the new image in both the main build process and the parallel build using xpanes.

A minor suggestion was made to add a comment for consistency, but otherwise, these changes are ready for approval.

Makefile (2)

28-28: LGTM: Variable reordering looks good.

The repositioning of the AGENT_NGT_IMAGE variable definition improves the overall organization of the Makefile. This change doesn't affect functionality and potentially enhances readability.


39-39: LGTM: New variable for example client image.

The addition of the EXAMPLE_CLIENT_IMAGE variable is consistent with the existing naming conventions and aligns with the PR objective of adding an example-client docker image. This change sets the groundwork for building and managing the new example client image within the project's build system.

dockers/example/client/Dockerfile Show resolved Hide resolved
Signed-off-by: vankichi <kyukawa315@gmail.com>
Copy link

cloudflare-workers-and-pages bot commented Oct 16, 2024

Deploying vald with  Cloudflare Pages  Cloudflare Pages

Latest commit: a160aa0
Status: ✅  Deploy successful!
Preview URL: https://6e4790fa.vald.pages.dev
Branch Preview URL: https://feature-dockers-add-example.vald.pages.dev

View logs

coderabbitai[bot]
coderabbitai bot previously approved these changes Oct 16, 2024
Signed-off-by: vankichi <kyukawa315@gmail.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 30a92c2 and 4e38a0a.

📒 Files selected for processing (1)
  • example/helm/values.yaml (1 hunks)
🧰 Additional context used

example/helm/values.yaml Show resolved Hide resolved
coderabbitai[bot]
coderabbitai bot previously approved these changes Oct 17, 2024
@vankichi vankichi requested a review from kpango October 17, 2024 04:42
Signed-off-by: vankichi <kyukawa315@gmail.com>
Signed-off-by: vankichi <kyukawa315@gmail.com>
@kmrmt kmrmt merged commit 325fb61 into main Oct 21, 2024
178 of 179 checks passed
@kmrmt kmrmt deleted the feature/dockers/add-example-client-docker-image branch October 21, 2024 07:56
vdaas-ci pushed a commit that referenced this pull request Oct 21, 2024
* 🐳 add example-client docker image

Signed-off-by: vankichi <kyukawa315@gmail.com>

* 🐛 fix build error

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

---------

Signed-off-by: vankichi <kyukawa315@gmail.com>
vankichi added a commit that referenced this pull request Oct 21, 2024
* 🐳 add example-client docker image



* 🐛 fix build error



* ♻️ fix



* ♻️ fix



* ♻️ fix



---------

Signed-off-by: vankichi <kyukawa315@gmail.com>
Co-authored-by: Kiichiro YUKAWA <kyukawa315@gmail.com>
takuyaymd pushed a commit to takuyaymd/vald that referenced this pull request Dec 2, 2024
* 🐳 add example-client docker image

Signed-off-by: vankichi <kyukawa315@gmail.com>

* 🐛 fix build error

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

---------

Signed-off-by: vankichi <kyukawa315@gmail.com>
takuyaymd pushed a commit to takuyaymd/vald that referenced this pull request Dec 2, 2024
* 🐳 add example-client docker image

Signed-off-by: vankichi <kyukawa315@gmail.com>

* 🐛 fix build error

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

* ♻️ fix

Signed-off-by: vankichi <kyukawa315@gmail.com>

---------

Signed-off-by: vankichi <kyukawa315@gmail.com>
This was referenced Dec 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants